1 Data preparation

1.1 Outline

  • Load scripts: loads libraries and useful scripts used in the analyses; all .R files contained in scripts at the root of the factory are automatically loaded

  • Load data: imports datasets, and may contain some ad hoc changes to the data such as specific data cleaning (not used in other reports), new variables used in the analyses, etc.

1.2 Load packages


library(reportfactory)
library(here)
library(rio) 
library(tidyverse)
library(incidence)
library(distcrete)
library(epitrix)
library(earlyR)
library(projections)
library(linelist)
library(remotes)
library(janitor)
library(kableExtra)
library(DT)
library(cyphr)
library(chngpt)
library(lubridate)
library(ggpubr)
library(ggnewscale)

1.3 Load scripts

These scripts will load:

  • all scripts stored as .R files inside /scripts/
  • all scripts stored as .R files inside /src/

These scripts also contain routines to access the latest clean encrypted data (see next section).


reportfactory::rfh_load_scripts()

1.4 Load clean data

We import the latest NHS pathways data:


x <- import_pathways() %>%
  as_tibble()
x
## # A tibble: 186,035 x 11
##    site_type date       sex   age   ccg_code ccg_name count postcode nhs_region
##    <chr>     <date>     <chr> <chr> <chr>    <chr>    <int> <chr>    <chr>     
##  1 111       2020-03-18 fema… miss… e380000… nhs_glo…     1 gl34fe   South West
##  2 111       2020-03-18 fema… miss… e380001… nhs_sou…     1 ne325nn  North Eas…
##  3 111       2020-03-18 fema… 0-18  e380000… nhs_air…     8 bd57jr   North Eas…
##  4 111       2020-03-18 fema… 0-18  e380000… nhs_ash…     7 tn254ab  South East
##  5 111       2020-03-18 fema… 0-18  e380000… nhs_bar…    35 rm13ae   London    
##  6 111       2020-03-18 fema… 0-18  e380000… nhs_bar…     9 n111np   London    
##  7 111       2020-03-18 fema… 0-18  e380000… nhs_bar…    11 s752py   North Eas…
##  8 111       2020-03-18 fema… 0-18  e380000… nhs_bas…    19 ss143hg  East of E…
##  9 111       2020-03-18 fema… 0-18  e380000… nhs_bas…     6 dn227xf  North Eas…
## 10 111       2020-03-18 fema… 0-18  e380000… nhs_bat…     9 ba25rp   South West
## # … with 186,025 more rows, and 2 more variables: day <int>, weekday <fct>

We also import demographics data for NHS regions in England, used later in our analysis:


path <- here::here("data", "csv", "nhs_region_population_2018.csv")
nhs_region_pop <- rio::import(path) %>%
  mutate(nhs_region = str_to_title(gsub("_"," ",nhs_region)))

nhs_region_pop$nhs_region <- gsub(" Of ", " of ", nhs_region_pop$nhs_region)
nhs_region_pop$nhs_region <- gsub(" And ", " and ", nhs_region_pop$nhs_region)
nhs_region_pop
##                  nhs_region variable      value
## 1                North West     0-18 0.22538599
## 2  North East and Yorkshire     0-18 0.21876449
## 3                  Midlands     0-18 0.22564656
## 4           East of England     0-18 0.22810783
## 5                    London     0-18 0.23764782
## 6                South East     0-18 0.22458811
## 7                South West     0-18 0.20799797
## 8                North West    19-69 0.64274078
## 9  North East and Yorkshire    19-69 0.64437753
## 10                 Midlands    19-69 0.63876675
## 11          East of England    19-69 0.63034229
## 12                   London    19-69 0.67820084
## 13               South East    19-69 0.63267336
## 14               South West    19-69 0.63176131
## 15               North West   70-120 0.13187323
## 16 North East and Yorkshire   70-120 0.13685797
## 17                 Midlands   70-120 0.13558669
## 18          East of England   70-120 0.14154988
## 19                   London   70-120 0.08415135
## 20               South East   70-120 0.14273853
## 21               South West   70-120 0.16024072

Finally, we import publically available deaths per NHS region:


dth <- import_deaths() %>%
  mutate(nhs_region = str_to_title(gsub("_"," ",nhs_region)))

#truncation to account for reporting delay
delay_max <- 21

dth$nhs_region <- gsub(" Of ", " of ", dth$nhs_region)
dth$nhs_region <- gsub(" And ", " and ", dth$nhs_region)
dth
##      date_report               nhs_region deaths
## 1     2020-03-01          East of England      0
## 2     2020-03-02          East of England      1
## 3     2020-03-03          East of England      0
## 4     2020-03-04          East of England      0
## 5     2020-03-05          East of England      0
## 6     2020-03-06          East of England      1
## 7     2020-03-07          East of England      0
## 8     2020-03-08          East of England      0
## 9     2020-03-09          East of England      1
## 10    2020-03-10          East of England      0
## 11    2020-03-11          East of England      0
## 12    2020-03-12          East of England      0
## 13    2020-03-13          East of England      1
## 14    2020-03-14          East of England      2
## 15    2020-03-15          East of England      2
## 16    2020-03-16          East of England      1
## 17    2020-03-17          East of England      1
## 18    2020-03-18          East of England      5
## 19    2020-03-19          East of England      4
## 20    2020-03-20          East of England      2
## 21    2020-03-21          East of England     11
## 22    2020-03-22          East of England     12
## 23    2020-03-23          East of England     11
## 24    2020-03-24          East of England     19
## 25    2020-03-25          East of England     26
## 26    2020-03-26          East of England     36
## 27    2020-03-27          East of England     38
## 28    2020-03-28          East of England     28
## 29    2020-03-29          East of England     43
## 30    2020-03-30          East of England     45
## 31    2020-03-31          East of England     70
## 32    2020-04-01          East of England     62
## 33    2020-04-02          East of England     65
## 34    2020-04-03          East of England     80
## 35    2020-04-04          East of England     71
## 36    2020-04-05          East of England     76
## 37    2020-04-06          East of England     71
## 38    2020-04-07          East of England     93
## 39    2020-04-08          East of England    111
## 40    2020-04-09          East of England     87
## 41    2020-04-10          East of England     74
## 42    2020-04-11          East of England     92
## 43    2020-04-12          East of England    100
## 44    2020-04-13          East of England     78
## 45    2020-04-14          East of England     61
## 46    2020-04-15          East of England     82
## 47    2020-04-16          East of England     74
## 48    2020-04-17          East of England     86
## 49    2020-04-18          East of England     64
## 50    2020-04-19          East of England     67
## 51    2020-04-20          East of England     67
## 52    2020-04-21          East of England     75
## 53    2020-04-22          East of England     67
## 54    2020-04-23          East of England     49
## 55    2020-04-24          East of England     66
## 56    2020-04-25          East of England     54
## 57    2020-04-26          East of England     48
## 58    2020-04-27          East of England     46
## 59    2020-04-28          East of England     58
## 60    2020-04-29          East of England     32
## 61    2020-04-30          East of England     45
## 62    2020-05-01          East of England     49
## 63    2020-05-02          East of England     29
## 64    2020-05-03          East of England     41
## 65    2020-05-04          East of England     19
## 66    2020-05-05          East of England     36
## 67    2020-05-06          East of England     31
## 68    2020-05-07          East of England     33
## 69    2020-05-08          East of England     33
## 70    2020-05-09          East of England     29
## 71    2020-05-10          East of England     22
## 72    2020-05-11          East of England     18
## 73    2020-05-12          East of England     21
## 74    2020-05-13          East of England     27
## 75    2020-05-14          East of England     26
## 76    2020-05-15          East of England     19
## 77    2020-05-16          East of England     26
## 78    2020-05-17          East of England     17
## 79    2020-05-18          East of England     25
## 80    2020-05-19          East of England     15
## 81    2020-05-20          East of England     26
## 82    2020-05-21          East of England     21
## 83    2020-05-22          East of England     13
## 84    2020-05-23          East of England     12
## 85    2020-05-24          East of England     17
## 86    2020-05-25          East of England     25
## 87    2020-05-26          East of England     14
## 88    2020-05-27          East of England     12
## 89    2020-05-28          East of England     17
## 90    2020-05-29          East of England     16
## 91    2020-05-30          East of England      9
## 92    2020-05-31          East of England      8
## 93    2020-06-01          East of England     17
## 94    2020-06-02          East of England     14
## 95    2020-06-03          East of England     10
## 96    2020-06-04          East of England      7
## 97    2020-06-05          East of England     14
## 98    2020-06-06          East of England      5
## 99    2020-06-07          East of England      9
## 100   2020-06-08          East of England      7
## 101   2020-06-09          East of England      6
## 102   2020-06-10          East of England      8
## 103   2020-06-11          East of England      1
## 104   2020-06-12          East of England      9
## 105   2020-06-13          East of England      5
## 106   2020-06-14          East of England      4
## 107   2020-06-15          East of England      8
## 108   2020-06-16          East of England      3
## 109   2020-06-17          East of England      7
## 110   2020-06-18          East of England      4
## 111   2020-06-19          East of England      7
## 112   2020-06-20          East of England      4
## 113   2020-06-21          East of England      3
## 114   2020-06-22          East of England      6
## 115   2020-06-23          East of England      5
## 116   2020-06-24          East of England      4
## 117   2020-06-25          East of England      1
## 118   2020-06-26          East of England      5
## 119   2020-06-27          East of England      6
## 120   2020-06-28          East of England      8
## 121   2020-06-29          East of England      4
## 122   2020-06-30          East of England      5
## 123   2020-07-01          East of England      2
## 124   2020-07-02          East of England      5
## 125   2020-07-03          East of England      0
## 126   2020-07-04          East of England      3
## 127   2020-07-05          East of England      1
## 128   2020-07-06          East of England      2
## 129   2020-07-07          East of England      2
## 130   2020-07-08          East of England      0
## 131   2020-07-09          East of England      8
## 132   2020-07-10          East of England      4
## 133   2020-07-11          East of England      2
## 134   2020-07-12          East of England      1
## 135   2020-07-13          East of England      7
## 136   2020-07-14          East of England      2
## 137   2020-07-15          East of England      0
## 138   2020-07-16          East of England      0
## 139   2020-07-17          East of England      0
## 140   2020-07-18          East of England      0
## 141   2020-07-19          East of England      1
## 142   2020-07-20          East of England      1
## 143   2020-07-21          East of England      0
## 144   2020-07-22          East of England      0
## 145   2020-03-01                   London      0
## 146   2020-03-02                   London      0
## 147   2020-03-03                   London      0
## 148   2020-03-04                   London      0
## 149   2020-03-05                   London      0
## 150   2020-03-06                   London      1
## 151   2020-03-07                   London      0
## 152   2020-03-08                   London      0
## 153   2020-03-09                   London      1
## 154   2020-03-10                   London      0
## 155   2020-03-11                   London      5
## 156   2020-03-12                   London      6
## 157   2020-03-13                   London     10
## 158   2020-03-14                   London     13
## 159   2020-03-15                   London      9
## 160   2020-03-16                   London     15
## 161   2020-03-17                   London     23
## 162   2020-03-18                   London     27
## 163   2020-03-19                   London     25
## 164   2020-03-20                   London     44
## 165   2020-03-21                   London     49
## 166   2020-03-22                   London     54
## 167   2020-03-23                   London     63
## 168   2020-03-24                   London     86
## 169   2020-03-25                   London    112
## 170   2020-03-26                   London    129
## 171   2020-03-27                   London    129
## 172   2020-03-28                   London    122
## 173   2020-03-29                   London    145
## 174   2020-03-30                   London    149
## 175   2020-03-31                   London    181
## 176   2020-04-01                   London    202
## 177   2020-04-02                   London    191
## 178   2020-04-03                   London    196
## 179   2020-04-04                   London    230
## 180   2020-04-05                   London    195
## 181   2020-04-06                   London    197
## 182   2020-04-07                   London    220
## 183   2020-04-08                   London    238
## 184   2020-04-09                   London    206
## 185   2020-04-10                   London    170
## 186   2020-04-11                   London    178
## 187   2020-04-12                   London    158
## 188   2020-04-13                   London    166
## 189   2020-04-14                   London    143
## 190   2020-04-15                   London    142
## 191   2020-04-16                   London    140
## 192   2020-04-17                   London    100
## 193   2020-04-18                   London    101
## 194   2020-04-19                   London    103
## 195   2020-04-20                   London     95
## 196   2020-04-21                   London     94
## 197   2020-04-22                   London    109
## 198   2020-04-23                   London     77
## 199   2020-04-24                   London     71
## 200   2020-04-25                   London     58
## 201   2020-04-26                   London     53
## 202   2020-04-27                   London     51
## 203   2020-04-28                   London     44
## 204   2020-04-29                   London     45
## 205   2020-04-30                   London     40
## 206   2020-05-01                   London     41
## 207   2020-05-02                   London     41
## 208   2020-05-03                   London     36
## 209   2020-05-04                   London     30
## 210   2020-05-05                   London     25
## 211   2020-05-06                   London     37
## 212   2020-05-07                   London     37
## 213   2020-05-08                   London     30
## 214   2020-05-09                   London     23
## 215   2020-05-10                   London     26
## 216   2020-05-11                   London     18
## 217   2020-05-12                   London     18
## 218   2020-05-13                   London     17
## 219   2020-05-14                   London     20
## 220   2020-05-15                   London     18
## 221   2020-05-16                   London     14
## 222   2020-05-17                   London     15
## 223   2020-05-18                   London     10
## 224   2020-05-19                   London     14
## 225   2020-05-20                   London     19
## 226   2020-05-21                   London     12
## 227   2020-05-22                   London     10
## 228   2020-05-23                   London      6
## 229   2020-05-24                   London      7
## 230   2020-05-25                   London      9
## 231   2020-05-26                   London     13
## 232   2020-05-27                   London      7
## 233   2020-05-28                   London      8
## 234   2020-05-29                   London      7
## 235   2020-05-30                   London     12
## 236   2020-05-31                   London      6
## 237   2020-06-01                   London     10
## 238   2020-06-02                   London      8
## 239   2020-06-03                   London      6
## 240   2020-06-04                   London      8
## 241   2020-06-05                   London      4
## 242   2020-06-06                   London      0
## 243   2020-06-07                   London      5
## 244   2020-06-08                   London      5
## 245   2020-06-09                   London      4
## 246   2020-06-10                   London      7
## 247   2020-06-11                   London      5
## 248   2020-06-12                   London      3
## 249   2020-06-13                   London      3
## 250   2020-06-14                   London      3
## 251   2020-06-15                   London      1
## 252   2020-06-16                   London      2
## 253   2020-06-17                   London      1
## 254   2020-06-18                   London      2
## 255   2020-06-19                   London      5
## 256   2020-06-20                   London      3
## 257   2020-06-21                   London      4
## 258   2020-06-22                   London      2
## 259   2020-06-23                   London      1
## 260   2020-06-24                   London      4
## 261   2020-06-25                   London      3
## 262   2020-06-26                   London      2
## 263   2020-06-27                   London      1
## 264   2020-06-28                   London      2
## 265   2020-06-29                   London      2
## 266   2020-06-30                   London      1
## 267   2020-07-01                   London      2
## 268   2020-07-02                   London      2
## 269   2020-07-03                   London      2
## 270   2020-07-04                   London      1
## 271   2020-07-05                   London      3
## 272   2020-07-06                   London      2
## 273   2020-07-07                   London      1
## 274   2020-07-08                   London      3
## 275   2020-07-09                   London      4
## 276   2020-07-10                   London      0
## 277   2020-07-11                   London      1
## 278   2020-07-12                   London      1
## 279   2020-07-13                   London      1
## 280   2020-07-14                   London      0
## 281   2020-07-15                   London      2
## 282   2020-07-16                   London      0
## 283   2020-07-17                   London      0
## 284   2020-07-18                   London      1
## 285   2020-07-19                   London      0
## 286   2020-07-20                   London      0
## 287   2020-07-21                   London      0
## 288   2020-07-22                   London      0
## 289   2020-03-01                 Midlands      0
## 290   2020-03-02                 Midlands      0
## 291   2020-03-03                 Midlands      1
## 292   2020-03-04                 Midlands      0
## 293   2020-03-05                 Midlands      0
## 294   2020-03-06                 Midlands      0
## 295   2020-03-07                 Midlands      0
## 296   2020-03-08                 Midlands      2
## 297   2020-03-09                 Midlands      1
## 298   2020-03-10                 Midlands      0
## 299   2020-03-11                 Midlands      2
## 300   2020-03-12                 Midlands      6
## 301   2020-03-13                 Midlands      5
## 302   2020-03-14                 Midlands      4
## 303   2020-03-15                 Midlands      5
## 304   2020-03-16                 Midlands     11
## 305   2020-03-17                 Midlands      8
## 306   2020-03-18                 Midlands     13
## 307   2020-03-19                 Midlands      8
## 308   2020-03-20                 Midlands     28
## 309   2020-03-21                 Midlands     13
## 310   2020-03-22                 Midlands     31
## 311   2020-03-23                 Midlands     33
## 312   2020-03-24                 Midlands     41
## 313   2020-03-25                 Midlands     48
## 314   2020-03-26                 Midlands     64
## 315   2020-03-27                 Midlands     72
## 316   2020-03-28                 Midlands     89
## 317   2020-03-29                 Midlands     92
## 318   2020-03-30                 Midlands     90
## 319   2020-03-31                 Midlands    123
## 320   2020-04-01                 Midlands    140
## 321   2020-04-02                 Midlands    142
## 322   2020-04-03                 Midlands    124
## 323   2020-04-04                 Midlands    151
## 324   2020-04-05                 Midlands    164
## 325   2020-04-06                 Midlands    140
## 326   2020-04-07                 Midlands    123
## 327   2020-04-08                 Midlands    186
## 328   2020-04-09                 Midlands    139
## 329   2020-04-10                 Midlands    127
## 330   2020-04-11                 Midlands    142
## 331   2020-04-12                 Midlands    139
## 332   2020-04-13                 Midlands    120
## 333   2020-04-14                 Midlands    116
## 334   2020-04-15                 Midlands    147
## 335   2020-04-16                 Midlands    102
## 336   2020-04-17                 Midlands    118
## 337   2020-04-18                 Midlands    115
## 338   2020-04-19                 Midlands     92
## 339   2020-04-20                 Midlands    107
## 340   2020-04-21                 Midlands     86
## 341   2020-04-22                 Midlands     78
## 342   2020-04-23                 Midlands    103
## 343   2020-04-24                 Midlands     79
## 344   2020-04-25                 Midlands     72
## 345   2020-04-26                 Midlands     81
## 346   2020-04-27                 Midlands     74
## 347   2020-04-28                 Midlands     68
## 348   2020-04-29                 Midlands     53
## 349   2020-04-30                 Midlands     56
## 350   2020-05-01                 Midlands     64
## 351   2020-05-02                 Midlands     51
## 352   2020-05-03                 Midlands     52
## 353   2020-05-04                 Midlands     61
## 354   2020-05-05                 Midlands     59
## 355   2020-05-06                 Midlands     59
## 356   2020-05-07                 Midlands     48
## 357   2020-05-08                 Midlands     34
## 358   2020-05-09                 Midlands     37
## 359   2020-05-10                 Midlands     42
## 360   2020-05-11                 Midlands     33
## 361   2020-05-12                 Midlands     45
## 362   2020-05-13                 Midlands     40
## 363   2020-05-14                 Midlands     38
## 364   2020-05-15                 Midlands     40
## 365   2020-05-16                 Midlands     34
## 366   2020-05-17                 Midlands     31
## 367   2020-05-18                 Midlands     36
## 368   2020-05-19                 Midlands     35
## 369   2020-05-20                 Midlands     36
## 370   2020-05-21                 Midlands     32
## 371   2020-05-22                 Midlands     27
## 372   2020-05-23                 Midlands     34
## 373   2020-05-24                 Midlands     20
## 374   2020-05-25                 Midlands     26
## 375   2020-05-26                 Midlands     33
## 376   2020-05-27                 Midlands     29
## 377   2020-05-28                 Midlands     28
## 378   2020-05-29                 Midlands     20
## 379   2020-05-30                 Midlands     21
## 380   2020-05-31                 Midlands     22
## 381   2020-06-01                 Midlands     20
## 382   2020-06-02                 Midlands     22
## 383   2020-06-03                 Midlands     24
## 384   2020-06-04                 Midlands     16
## 385   2020-06-05                 Midlands     21
## 386   2020-06-06                 Midlands     20
## 387   2020-06-07                 Midlands     17
## 388   2020-06-08                 Midlands     16
## 389   2020-06-09                 Midlands     18
## 390   2020-06-10                 Midlands     15
## 391   2020-06-11                 Midlands     13
## 392   2020-06-12                 Midlands     12
## 393   2020-06-13                 Midlands      6
## 394   2020-06-14                 Midlands     18
## 395   2020-06-15                 Midlands     12
## 396   2020-06-16                 Midlands     15
## 397   2020-06-17                 Midlands     11
## 398   2020-06-18                 Midlands     15
## 399   2020-06-19                 Midlands     10
## 400   2020-06-20                 Midlands     15
## 401   2020-06-21                 Midlands     14
## 402   2020-06-22                 Midlands     14
## 403   2020-06-23                 Midlands     16
## 404   2020-06-24                 Midlands     15
## 405   2020-06-25                 Midlands     18
## 406   2020-06-26                 Midlands      5
## 407   2020-06-27                 Midlands      5
## 408   2020-06-28                 Midlands      7
## 409   2020-06-29                 Midlands      6
## 410   2020-06-30                 Midlands      6
## 411   2020-07-01                 Midlands      7
## 412   2020-07-02                 Midlands      9
## 413   2020-07-03                 Midlands      3
## 414   2020-07-04                 Midlands      4
## 415   2020-07-05                 Midlands      6
## 416   2020-07-06                 Midlands      5
## 417   2020-07-07                 Midlands      3
## 418   2020-07-08                 Midlands      5
## 419   2020-07-09                 Midlands      8
## 420   2020-07-10                 Midlands      3
## 421   2020-07-11                 Midlands      0
## 422   2020-07-12                 Midlands      5
## 423   2020-07-13                 Midlands      1
## 424   2020-07-14                 Midlands      1
## 425   2020-07-15                 Midlands      6
## 426   2020-07-16                 Midlands      2
## 427   2020-07-17                 Midlands      1
## 428   2020-07-18                 Midlands      2
## 429   2020-07-19                 Midlands      3
## 430   2020-07-20                 Midlands      1
## 431   2020-07-21                 Midlands      0
## 432   2020-07-22                 Midlands      0
## 433   2020-03-01 North East and Yorkshire      0
## 434   2020-03-02 North East and Yorkshire      0
## 435   2020-03-03 North East and Yorkshire      0
## 436   2020-03-04 North East and Yorkshire      0
## 437   2020-03-05 North East and Yorkshire      0
## 438   2020-03-06 North East and Yorkshire      0
## 439   2020-03-07 North East and Yorkshire      0
## 440   2020-03-08 North East and Yorkshire      0
## 441   2020-03-09 North East and Yorkshire      0
## 442   2020-03-10 North East and Yorkshire      0
## 443   2020-03-11 North East and Yorkshire      0
## 444   2020-03-12 North East and Yorkshire      0
## 445   2020-03-13 North East and Yorkshire      0
## 446   2020-03-14 North East and Yorkshire      0
## 447   2020-03-15 North East and Yorkshire      2
## 448   2020-03-16 North East and Yorkshire      3
## 449   2020-03-17 North East and Yorkshire      1
## 450   2020-03-18 North East and Yorkshire      2
## 451   2020-03-19 North East and Yorkshire      6
## 452   2020-03-20 North East and Yorkshire      5
## 453   2020-03-21 North East and Yorkshire      6
## 454   2020-03-22 North East and Yorkshire      7
## 455   2020-03-23 North East and Yorkshire      9
## 456   2020-03-24 North East and Yorkshire      8
## 457   2020-03-25 North East and Yorkshire     18
## 458   2020-03-26 North East and Yorkshire     21
## 459   2020-03-27 North East and Yorkshire     28
## 460   2020-03-28 North East and Yorkshire     35
## 461   2020-03-29 North East and Yorkshire     38
## 462   2020-03-30 North East and Yorkshire     64
## 463   2020-03-31 North East and Yorkshire     60
## 464   2020-04-01 North East and Yorkshire     67
## 465   2020-04-02 North East and Yorkshire     75
## 466   2020-04-03 North East and Yorkshire    100
## 467   2020-04-04 North East and Yorkshire    105
## 468   2020-04-05 North East and Yorkshire     92
## 469   2020-04-06 North East and Yorkshire     96
## 470   2020-04-07 North East and Yorkshire    102
## 471   2020-04-08 North East and Yorkshire    107
## 472   2020-04-09 North East and Yorkshire    111
## 473   2020-04-10 North East and Yorkshire    117
## 474   2020-04-11 North East and Yorkshire     98
## 475   2020-04-12 North East and Yorkshire     84
## 476   2020-04-13 North East and Yorkshire     94
## 477   2020-04-14 North East and Yorkshire    107
## 478   2020-04-15 North East and Yorkshire     96
## 479   2020-04-16 North East and Yorkshire    103
## 480   2020-04-17 North East and Yorkshire     88
## 481   2020-04-18 North East and Yorkshire     95
## 482   2020-04-19 North East and Yorkshire     88
## 483   2020-04-20 North East and Yorkshire    100
## 484   2020-04-21 North East and Yorkshire     76
## 485   2020-04-22 North East and Yorkshire     84
## 486   2020-04-23 North East and Yorkshire     63
## 487   2020-04-24 North East and Yorkshire     72
## 488   2020-04-25 North East and Yorkshire     69
## 489   2020-04-26 North East and Yorkshire     65
## 490   2020-04-27 North East and Yorkshire     65
## 491   2020-04-28 North East and Yorkshire     57
## 492   2020-04-29 North East and Yorkshire     69
## 493   2020-04-30 North East and Yorkshire     57
## 494   2020-05-01 North East and Yorkshire     64
## 495   2020-05-02 North East and Yorkshire     48
## 496   2020-05-03 North East and Yorkshire     40
## 497   2020-05-04 North East and Yorkshire     49
## 498   2020-05-05 North East and Yorkshire     40
## 499   2020-05-06 North East and Yorkshire     51
## 500   2020-05-07 North East and Yorkshire     45
## 501   2020-05-08 North East and Yorkshire     42
## 502   2020-05-09 North East and Yorkshire     44
## 503   2020-05-10 North East and Yorkshire     40
## 504   2020-05-11 North East and Yorkshire     29
## 505   2020-05-12 North East and Yorkshire     27
## 506   2020-05-13 North East and Yorkshire     28
## 507   2020-05-14 North East and Yorkshire     31
## 508   2020-05-15 North East and Yorkshire     32
## 509   2020-05-16 North East and Yorkshire     35
## 510   2020-05-17 North East and Yorkshire     26
## 511   2020-05-18 North East and Yorkshire     30
## 512   2020-05-19 North East and Yorkshire     27
## 513   2020-05-20 North East and Yorkshire     22
## 514   2020-05-21 North East and Yorkshire     33
## 515   2020-05-22 North East and Yorkshire     22
## 516   2020-05-23 North East and Yorkshire     18
## 517   2020-05-24 North East and Yorkshire     26
## 518   2020-05-25 North East and Yorkshire     21
## 519   2020-05-26 North East and Yorkshire     21
## 520   2020-05-27 North East and Yorkshire     22
## 521   2020-05-28 North East and Yorkshire     21
## 522   2020-05-29 North East and Yorkshire     25
## 523   2020-05-30 North East and Yorkshire     20
## 524   2020-05-31 North East and Yorkshire     20
## 525   2020-06-01 North East and Yorkshire     17
## 526   2020-06-02 North East and Yorkshire     23
## 527   2020-06-03 North East and Yorkshire     23
## 528   2020-06-04 North East and Yorkshire     17
## 529   2020-06-05 North East and Yorkshire     18
## 530   2020-06-06 North East and Yorkshire     21
## 531   2020-06-07 North East and Yorkshire     14
## 532   2020-06-08 North East and Yorkshire     11
## 533   2020-06-09 North East and Yorkshire     12
## 534   2020-06-10 North East and Yorkshire     19
## 535   2020-06-11 North East and Yorkshire      7
## 536   2020-06-12 North East and Yorkshire      9
## 537   2020-06-13 North East and Yorkshire     10
## 538   2020-06-14 North East and Yorkshire     11
## 539   2020-06-15 North East and Yorkshire      9
## 540   2020-06-16 North East and Yorkshire     10
## 541   2020-06-17 North East and Yorkshire      9
## 542   2020-06-18 North East and Yorkshire     11
## 543   2020-06-19 North East and Yorkshire      6
## 544   2020-06-20 North East and Yorkshire      5
## 545   2020-06-21 North East and Yorkshire      4
## 546   2020-06-22 North East and Yorkshire      7
## 547   2020-06-23 North East and Yorkshire      7
## 548   2020-06-24 North East and Yorkshire     10
## 549   2020-06-25 North East and Yorkshire      4
## 550   2020-06-26 North East and Yorkshire      7
## 551   2020-06-27 North East and Yorkshire      3
## 552   2020-06-28 North East and Yorkshire      5
## 553   2020-06-29 North East and Yorkshire      2
## 554   2020-06-30 North East and Yorkshire      5
## 555   2020-07-01 North East and Yorkshire      1
## 556   2020-07-02 North East and Yorkshire      4
## 557   2020-07-03 North East and Yorkshire      4
## 558   2020-07-04 North East and Yorkshire      4
## 559   2020-07-05 North East and Yorkshire      2
## 560   2020-07-06 North East and Yorkshire      2
## 561   2020-07-07 North East and Yorkshire      3
## 562   2020-07-08 North East and Yorkshire      3
## 563   2020-07-09 North East and Yorkshire      0
## 564   2020-07-10 North East and Yorkshire      3
## 565   2020-07-11 North East and Yorkshire      1
## 566   2020-07-12 North East and Yorkshire      4
## 567   2020-07-13 North East and Yorkshire      1
## 568   2020-07-14 North East and Yorkshire      1
## 569   2020-07-15 North East and Yorkshire      2
## 570   2020-07-16 North East and Yorkshire      2
## 571   2020-07-17 North East and Yorkshire      1
## 572   2020-07-18 North East and Yorkshire      2
## 573   2020-07-19 North East and Yorkshire      2
## 574   2020-07-20 North East and Yorkshire      1
## 575   2020-07-21 North East and Yorkshire      1
## 576   2020-07-22 North East and Yorkshire      1
## 577   2020-03-01               North West      0
## 578   2020-03-02               North West      0
## 579   2020-03-03               North West      0
## 580   2020-03-04               North West      0
## 581   2020-03-05               North West      1
## 582   2020-03-06               North West      0
## 583   2020-03-07               North West      0
## 584   2020-03-08               North West      1
## 585   2020-03-09               North West      0
## 586   2020-03-10               North West      0
## 587   2020-03-11               North West      0
## 588   2020-03-12               North West      2
## 589   2020-03-13               North West      3
## 590   2020-03-14               North West      1
## 591   2020-03-15               North West      4
## 592   2020-03-16               North West      2
## 593   2020-03-17               North West      4
## 594   2020-03-18               North West      6
## 595   2020-03-19               North West      7
## 596   2020-03-20               North West     10
## 597   2020-03-21               North West     11
## 598   2020-03-22               North West     13
## 599   2020-03-23               North West     15
## 600   2020-03-24               North West     21
## 601   2020-03-25               North West     21
## 602   2020-03-26               North West     29
## 603   2020-03-27               North West     36
## 604   2020-03-28               North West     28
## 605   2020-03-29               North West     46
## 606   2020-03-30               North West     67
## 607   2020-03-31               North West     52
## 608   2020-04-01               North West     86
## 609   2020-04-02               North West     96
## 610   2020-04-03               North West     95
## 611   2020-04-04               North West     98
## 612   2020-04-05               North West    102
## 613   2020-04-06               North West    100
## 614   2020-04-07               North West    135
## 615   2020-04-08               North West    127
## 616   2020-04-09               North West    119
## 617   2020-04-10               North West    117
## 618   2020-04-11               North West    138
## 619   2020-04-12               North West    125
## 620   2020-04-13               North West    129
## 621   2020-04-14               North West    131
## 622   2020-04-15               North West    114
## 623   2020-04-16               North West    135
## 624   2020-04-17               North West     98
## 625   2020-04-18               North West    113
## 626   2020-04-19               North West     71
## 627   2020-04-20               North West     83
## 628   2020-04-21               North West     76
## 629   2020-04-22               North West     86
## 630   2020-04-23               North West     85
## 631   2020-04-24               North West     66
## 632   2020-04-25               North West     66
## 633   2020-04-26               North West     55
## 634   2020-04-27               North West     54
## 635   2020-04-28               North West     57
## 636   2020-04-29               North West     63
## 637   2020-04-30               North West     59
## 638   2020-05-01               North West     45
## 639   2020-05-02               North West     56
## 640   2020-05-03               North West     55
## 641   2020-05-04               North West     48
## 642   2020-05-05               North West     48
## 643   2020-05-06               North West     44
## 644   2020-05-07               North West     49
## 645   2020-05-08               North West     42
## 646   2020-05-09               North West     31
## 647   2020-05-10               North West     42
## 648   2020-05-11               North West     35
## 649   2020-05-12               North West     38
## 650   2020-05-13               North West     25
## 651   2020-05-14               North West     26
## 652   2020-05-15               North West     33
## 653   2020-05-16               North West     32
## 654   2020-05-17               North West     24
## 655   2020-05-18               North West     31
## 656   2020-05-19               North West     35
## 657   2020-05-20               North West     27
## 658   2020-05-21               North West     27
## 659   2020-05-22               North West     26
## 660   2020-05-23               North West     31
## 661   2020-05-24               North West     26
## 662   2020-05-25               North West     31
## 663   2020-05-26               North West     27
## 664   2020-05-27               North West     27
## 665   2020-05-28               North West     28
## 666   2020-05-29               North West     20
## 667   2020-05-30               North West     19
## 668   2020-05-31               North West     13
## 669   2020-06-01               North West     12
## 670   2020-06-02               North West     27
## 671   2020-06-03               North West     22
## 672   2020-06-04               North West     22
## 673   2020-06-05               North West     16
## 674   2020-06-06               North West     26
## 675   2020-06-07               North West     20
## 676   2020-06-08               North West     23
## 677   2020-06-09               North West     17
## 678   2020-06-10               North West     16
## 679   2020-06-11               North West     16
## 680   2020-06-12               North West     11
## 681   2020-06-13               North West     10
## 682   2020-06-14               North West     15
## 683   2020-06-15               North West     16
## 684   2020-06-16               North West     15
## 685   2020-06-17               North West     13
## 686   2020-06-18               North West     14
## 687   2020-06-19               North West      7
## 688   2020-06-20               North West     11
## 689   2020-06-21               North West      8
## 690   2020-06-22               North West     11
## 691   2020-06-23               North West     13
## 692   2020-06-24               North West     13
## 693   2020-06-25               North West     15
## 694   2020-06-26               North West      6
## 695   2020-06-27               North West      7
## 696   2020-06-28               North West      9
## 697   2020-06-29               North West      9
## 698   2020-06-30               North West      7
## 699   2020-07-01               North West      3
## 700   2020-07-02               North West      6
## 701   2020-07-03               North West      7
## 702   2020-07-04               North West      4
## 703   2020-07-05               North West      6
## 704   2020-07-06               North West      9
## 705   2020-07-07               North West      8
## 706   2020-07-08               North West      5
## 707   2020-07-09               North West     10
## 708   2020-07-10               North West      2
## 709   2020-07-11               North West      4
## 710   2020-07-12               North West      0
## 711   2020-07-13               North West      6
## 712   2020-07-14               North West      4
## 713   2020-07-15               North West      5
## 714   2020-07-16               North West      2
## 715   2020-07-17               North West      4
## 716   2020-07-18               North West      2
## 717   2020-07-19               North West      1
## 718   2020-07-20               North West      0
## 719   2020-07-21               North West      0
## 720   2020-07-22               North West      0
## 721   2020-03-01               South East      0
## 722   2020-03-02               South East      0
## 723   2020-03-03               South East      1
## 724   2020-03-04               South East      0
## 725   2020-03-05               South East      1
## 726   2020-03-06               South East      0
## 727   2020-03-07               South East      0
## 728   2020-03-08               South East      1
## 729   2020-03-09               South East      1
## 730   2020-03-10               South East      1
## 731   2020-03-11               South East      1
## 732   2020-03-12               South East      0
## 733   2020-03-13               South East      1
## 734   2020-03-14               South East      1
## 735   2020-03-15               South East      5
## 736   2020-03-16               South East      8
## 737   2020-03-17               South East      7
## 738   2020-03-18               South East     10
## 739   2020-03-19               South East      9
## 740   2020-03-20               South East     13
## 741   2020-03-21               South East      7
## 742   2020-03-22               South East     25
## 743   2020-03-23               South East     20
## 744   2020-03-24               South East     22
## 745   2020-03-25               South East     29
## 746   2020-03-26               South East     35
## 747   2020-03-27               South East     34
## 748   2020-03-28               South East     36
## 749   2020-03-29               South East     55
## 750   2020-03-30               South East     58
## 751   2020-03-31               South East     65
## 752   2020-04-01               South East     66
## 753   2020-04-02               South East     55
## 754   2020-04-03               South East     72
## 755   2020-04-04               South East     80
## 756   2020-04-05               South East     82
## 757   2020-04-06               South East     88
## 758   2020-04-07               South East    100
## 759   2020-04-08               South East     83
## 760   2020-04-09               South East    104
## 761   2020-04-10               South East     88
## 762   2020-04-11               South East     88
## 763   2020-04-12               South East     88
## 764   2020-04-13               South East     84
## 765   2020-04-14               South East     65
## 766   2020-04-15               South East     72
## 767   2020-04-16               South East     56
## 768   2020-04-17               South East     86
## 769   2020-04-18               South East     57
## 770   2020-04-19               South East     70
## 771   2020-04-20               South East     87
## 772   2020-04-21               South East     51
## 773   2020-04-22               South East     54
## 774   2020-04-23               South East     57
## 775   2020-04-24               South East     64
## 776   2020-04-25               South East     51
## 777   2020-04-26               South East     51
## 778   2020-04-27               South East     41
## 779   2020-04-28               South East     40
## 780   2020-04-29               South East     47
## 781   2020-04-30               South East     29
## 782   2020-05-01               South East     37
## 783   2020-05-02               South East     36
## 784   2020-05-03               South East     17
## 785   2020-05-04               South East     35
## 786   2020-05-05               South East     29
## 787   2020-05-06               South East     25
## 788   2020-05-07               South East     27
## 789   2020-05-08               South East     26
## 790   2020-05-09               South East     28
## 791   2020-05-10               South East     19
## 792   2020-05-11               South East     25
## 793   2020-05-12               South East     27
## 794   2020-05-13               South East     18
## 795   2020-05-14               South East     32
## 796   2020-05-15               South East     25
## 797   2020-05-16               South East     22
## 798   2020-05-17               South East     18
## 799   2020-05-18               South East     22
## 800   2020-05-19               South East     12
## 801   2020-05-20               South East     22
## 802   2020-05-21               South East     15
## 803   2020-05-22               South East     17
## 804   2020-05-23               South East     21
## 805   2020-05-24               South East     17
## 806   2020-05-25               South East     13
## 807   2020-05-26               South East     19
## 808   2020-05-27               South East     18
## 809   2020-05-28               South East     12
## 810   2020-05-29               South East     21
## 811   2020-05-30               South East      8
## 812   2020-05-31               South East     12
## 813   2020-06-01               South East     11
## 814   2020-06-02               South East     13
## 815   2020-06-03               South East     18
## 816   2020-06-04               South East     11
## 817   2020-06-05               South East     11
## 818   2020-06-06               South East     10
## 819   2020-06-07               South East     12
## 820   2020-06-08               South East      8
## 821   2020-06-09               South East     10
## 822   2020-06-10               South East     11
## 823   2020-06-11               South East      5
## 824   2020-06-12               South East      6
## 825   2020-06-13               South East      7
## 826   2020-06-14               South East      7
## 827   2020-06-15               South East      8
## 828   2020-06-16               South East     13
## 829   2020-06-17               South East      9
## 830   2020-06-18               South East      4
## 831   2020-06-19               South East      7
## 832   2020-06-20               South East      5
## 833   2020-06-21               South East      3
## 834   2020-06-22               South East      2
## 835   2020-06-23               South East      8
## 836   2020-06-24               South East      7
## 837   2020-06-25               South East      5
## 838   2020-06-26               South East      8
## 839   2020-06-27               South East      8
## 840   2020-06-28               South East      6
## 841   2020-06-29               South East      5
## 842   2020-06-30               South East      5
## 843   2020-07-01               South East      2
## 844   2020-07-02               South East      7
## 845   2020-07-03               South East      3
## 846   2020-07-04               South East      6
## 847   2020-07-05               South East      4
## 848   2020-07-06               South East      3
## 849   2020-07-07               South East      6
## 850   2020-07-08               South East      3
## 851   2020-07-09               South East      7
## 852   2020-07-10               South East      3
## 853   2020-07-11               South East      2
## 854   2020-07-12               South East      4
## 855   2020-07-13               South East      4
## 856   2020-07-14               South East      4
## 857   2020-07-15               South East      4
## 858   2020-07-16               South East      3
## 859   2020-07-17               South East      1
## 860   2020-07-18               South East      4
## 861   2020-07-19               South East      2
## 862   2020-07-20               South East      2
## 863   2020-07-21               South East      2
## 864   2020-07-22               South East      0
## 865   2020-03-01               South West      0
## 866   2020-03-02               South West      0
## 867   2020-03-03               South West      0
## 868   2020-03-04               South West      0
## 869   2020-03-05               South West      0
## 870   2020-03-06               South West      0
## 871   2020-03-07               South West      0
## 872   2020-03-08               South West      0
## 873   2020-03-09               South West      0
## 874   2020-03-10               South West      0
## 875   2020-03-11               South West      1
## 876   2020-03-12               South West      0
## 877   2020-03-13               South West      0
## 878   2020-03-14               South West      1
## 879   2020-03-15               South West      0
## 880   2020-03-16               South West      0
## 881   2020-03-17               South West      2
## 882   2020-03-18               South West      2
## 883   2020-03-19               South West      4
## 884   2020-03-20               South West      3
## 885   2020-03-21               South West      6
## 886   2020-03-22               South West      7
## 887   2020-03-23               South West      8
## 888   2020-03-24               South West      7
## 889   2020-03-25               South West      9
## 890   2020-03-26               South West     11
## 891   2020-03-27               South West     13
## 892   2020-03-28               South West     21
## 893   2020-03-29               South West     18
## 894   2020-03-30               South West     23
## 895   2020-03-31               South West     23
## 896   2020-04-01               South West     21
## 897   2020-04-02               South West     23
## 898   2020-04-03               South West     30
## 899   2020-04-04               South West     42
## 900   2020-04-05               South West     32
## 901   2020-04-06               South West     34
## 902   2020-04-07               South West     39
## 903   2020-04-08               South West     47
## 904   2020-04-09               South West     24
## 905   2020-04-10               South West     46
## 906   2020-04-11               South West     43
## 907   2020-04-12               South West     23
## 908   2020-04-13               South West     27
## 909   2020-04-14               South West     24
## 910   2020-04-15               South West     32
## 911   2020-04-16               South West     29
## 912   2020-04-17               South West     33
## 913   2020-04-18               South West     25
## 914   2020-04-19               South West     31
## 915   2020-04-20               South West     26
## 916   2020-04-21               South West     26
## 917   2020-04-22               South West     23
## 918   2020-04-23               South West     17
## 919   2020-04-24               South West     19
## 920   2020-04-25               South West     15
## 921   2020-04-26               South West     27
## 922   2020-04-27               South West     13
## 923   2020-04-28               South West     17
## 924   2020-04-29               South West     15
## 925   2020-04-30               South West     26
## 926   2020-05-01               South West      6
## 927   2020-05-02               South West      7
## 928   2020-05-03               South West     10
## 929   2020-05-04               South West     17
## 930   2020-05-05               South West     14
## 931   2020-05-06               South West     19
## 932   2020-05-07               South West     16
## 933   2020-05-08               South West      6
## 934   2020-05-09               South West     11
## 935   2020-05-10               South West      5
## 936   2020-05-11               South West      8
## 937   2020-05-12               South West      7
## 938   2020-05-13               South West      7
## 939   2020-05-14               South West      6
## 940   2020-05-15               South West      4
## 941   2020-05-16               South West      4
## 942   2020-05-17               South West      6
## 943   2020-05-18               South West      4
## 944   2020-05-19               South West      6
## 945   2020-05-20               South West      1
## 946   2020-05-21               South West      9
## 947   2020-05-22               South West      6
## 948   2020-05-23               South West      6
## 949   2020-05-24               South West      3
## 950   2020-05-25               South West      8
## 951   2020-05-26               South West     11
## 952   2020-05-27               South West      5
## 953   2020-05-28               South West     10
## 954   2020-05-29               South West      7
## 955   2020-05-30               South West      3
## 956   2020-05-31               South West      2
## 957   2020-06-01               South West      7
## 958   2020-06-02               South West      2
## 959   2020-06-03               South West      7
## 960   2020-06-04               South West      2
## 961   2020-06-05               South West      2
## 962   2020-06-06               South West      1
## 963   2020-06-07               South West      3
## 964   2020-06-08               South West      3
## 965   2020-06-09               South West      0
## 966   2020-06-10               South West      1
## 967   2020-06-11               South West      2
## 968   2020-06-12               South West      2
## 969   2020-06-13               South West      2
## 970   2020-06-14               South West      0
## 971   2020-06-15               South West      2
## 972   2020-06-16               South West      2
## 973   2020-06-17               South West      0
## 974   2020-06-18               South West      0
## 975   2020-06-19               South West      0
## 976   2020-06-20               South West      2
## 977   2020-06-21               South West      0
## 978   2020-06-22               South West      1
## 979   2020-06-23               South West      1
## 980   2020-06-24               South West      1
## 981   2020-06-25               South West      0
## 982   2020-06-26               South West      3
## 983   2020-06-27               South West      0
## 984   2020-06-28               South West      0
## 985   2020-06-29               South West      1
## 986   2020-06-30               South West      0
## 987   2020-07-01               South West      0
## 988   2020-07-02               South West      0
## 989   2020-07-03               South West      0
## 990   2020-07-04               South West      0
## 991   2020-07-05               South West      1
## 992   2020-07-06               South West      0
## 993   2020-07-07               South West      0
## 994   2020-07-08               South West      2
## 995   2020-07-09               South West      0
## 996   2020-07-10               South West      1
## 997   2020-07-11               South West      0
## 998   2020-07-12               South West      0
## 999   2020-07-13               South West      1
## 1000  2020-07-14               South West      0
## 1001  2020-07-15               South West      0
## 1002  2020-07-16               South West      0
## 1003  2020-07-17               South West      1
## 1004  2020-07-18               South West      0
## 1005  2020-07-19               South West      0
## 1006  2020-07-20               South West      0
## 1007  2020-07-21               South West      0
## 1008  2020-07-22               South West      0

1.5 Completion date

We extract the completion date from the NHS Pathways file timestamp:


database_date <- attr(x, "timestamp")
database_date
## [1] "2020-07-23"

The completion date of the NHS Pathways data is Thursday 23 Jul 2020.

1.6 Auxiliary functions

These are functions which will be used further in the analyses.

Function to estimate the generalised R-squared as the proportion of deviance explained by a given model:


## Function to calculate R2 for Poisson model
## not adjusted for model complexity but all models have the same DF here

Rsq <- function(x) {
  1 - (x$deviance / x$null.deviance)
}

Function to extract growth rates per region as well as halving times, and the associated 95% confidence intervals:


## function to extract the coefficients, find the level of the intercept,
## reconstruct the values of r, get confidence intervals

get_r <- function(model) {
  ##  extract coefficients and conf int
  out <- data.frame(r = coef(model))  %>%
    rownames_to_column("var") %>% 
    cbind(confint(model)) %>%
    filter(!grepl("day_of_week", var)) %>% 
    filter(grepl("day", var)) %>%
    rename(lower_95 = "2.5 %",
           upper_95 = "97.5 %") %>%
    mutate(var = sub("day:", "", var))
  
  ## reconstruct values: intercept + region-coefficient
  for (i in 2:nrow(out)) {
    out[i, -1] <- out[1, -1] + out[i, -1]
  }
  
  ## find the name of the intercept, restore regions names
  out <- out %>%
    mutate(nhs_region = model$xlevels$nhs_region) %>%
    select(nhs_region, everything(), -var)
  
  ## find halving times
  halving <- log(0.5) / out[,-1] %>%
    rename(halving_t = r,
           halving_t_lower_95 = lower_95,
           halving_t_upper_95 = upper_95)
  
  ## set halving times with exclusion intervals to NA
  no_halving <- out$lower_95 < 0 & out$upper_95 > 0
  halving[no_halving, ] <- NA_real_
  
  ## return all data
  cbind(out, halving)
  
}

Functions used in the correlation analysis between NHS Pathways reports and deaths:

## Function to calculate Pearson's correlation between deaths and lagged
## reports. Note that `pearson` can be replaced with `spearman` for rank
## correlation.

getcor <- function(x, ndx) {
  return(cor(x$deaths[ndx],
             x$note_lag[ndx],
             use = "complete.obs",
             method = "pearson"))
}

## Catch if sample size throws an error
getcor2 <- possibly(getcor, otherwise = NA)

getboot <- function(x) {
  result <- boot::boot.ci(boot::boot(x, getcor2, R = 1000), 
                           type = "bca")
  return(data.frame(n = sum(!is.na(x$note_lag) & !is.na(x$deaths)),
                    r = result$t0,
                    r_low = result$bca[4],
                    r_hi = result$bca[5]))
}

Function to classify the day of the week into weekend, Monday, and the rest:


## Fn to add day of week
day_of_week <- function(df) {
  df %>% 
    dplyr::mutate(day_of_week = lubridate::wday(date, label = TRUE)) %>% 
    dplyr::mutate(day_of_week = dplyr::case_when(
      day_of_week %in% c("Sat", "Sun") ~ "weekend",
      day_of_week %in% c("Mon") ~ "monday",
      !(day_of_week %in% c("Sat", "Sun", "Mon")) ~ "rest_of_week"
    ) %>% 
      factor(levels = c("rest_of_week", "monday", "weekend")))
}

Custom color palettes, color scales, and vectors of colors:


pal <- c("#006212",
         "#ae3cab",
         "#00db90",
         "#960c00",
         "#55aaff",
         "#ff7e78",
         "#00388d")

age.pal <- viridis::viridis(3,begin = 0.1, end = 0.7)

3 Comparison with deaths time series

3.1 Outline

We want to explore the correlation between NHS Pathways reports and deaths, and assess the potential for reports to be used as an early warning system for disease resurgence.

Death data are publically available. We truncate the time series to avoid bias from reporting delay - we assume a conservative delay of three weeks.

3.2 Lagged correlation

We calculate Pearson’s correlation coefficient between deaths and NHS Pathways notifications using different lags. Confidence intervals are obtained using bootstrap. Note that results were also confirmed using Spearman’s rank correlation.

First we join the NHS Pathways and death data, and aggregate over all England:

## truncate death data for reporting delay
trunc_date <- max(dth$date_report) - delay_max

dth_trunc <- dth %>%
  rename(date = date_report) %>%
  filter(date <= trunc_date) 

## join with notification data
all_data <- x %>% 
  filter(!is.na(nhs_region)) %>%
  group_by(date, nhs_region) %>%
  summarise(count = sum(count, na.rm = T)) %>%
  ungroup %>%
  inner_join(dth_trunc,
             by = c("date","nhs_region"))

all_tot <- all_data %>%
  group_by(date) %>%
  summarise(count = sum(count, na.rm = TRUE),
            deaths = sum(deaths, na.rm = TRUE)) 

We calculate correlation with lagged NHS Pathways reports from 0 to 30 days behind deaths:


## Calculate all correlations + bootstrap CIs
lag_cor <- data.frame()
for (i in 0:30) {
  
  ## lag reports
  summary <- all_tot %>% 
    mutate(note_lag = lag(count, i)) %>%
    ## calculate rank correlation and bootstrap CI
    getboot(.) %>%
    mutate(lag = i)

  lag_cor <- bind_rows(lag_cor, summary)
}

cor_vs_lag <- ggplot(lag_cor, aes(lag, r)) +
  theme_bw() +
  geom_ribbon(aes(ymin = r_low, ymax = r_hi), alpha = 0.2) +
  geom_hline(yintercept = 0, lty = "longdash") +
  geom_point() +
  geom_line() +
  labs(x = "Lag between NHS pathways and death data (days)",
       y = "Pearson's correlation") +
  large_txt
cor_vs_lag


l_opt <- which.max(lag_cor$r)

This analysis suggests that the best lag is 23 days. We then compare and plot the number of deaths reported against the number of NHS Pathways reports lagged by 23 days.


all_tot <- all_tot %>%
  rename(date_death = date) %>%
  mutate(note_lag = lag(count, lag_cor$lag[l_opt]),
         note_lag_c = (note_lag - mean(note_lag, na.rm = T)),
         date_note = lag(date_death,16))

lag_mod <- glm(deaths ~ note_lag, data = all_tot, family = "quasipoisson")

summary(lag_mod)
## 
## Call:
## glm(formula = deaths ~ note_lag, family = "quasipoisson", data = all_tot)
## 
## Deviance Residuals: 
##      Min        1Q    Median        3Q       Max  
## -14.1341   -4.9640   -0.5784    3.8745    8.9805  
## 
## Coefficients:
##              Estimate Std. Error t value Pr(>|t|)    
## (Intercept) 4.520e+00  6.771e-02   66.76   <2e-16 ***
## note_lag    1.516e-05  7.063e-07   21.46   <2e-16 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for quasipoisson family taken to be 27.57536)
## 
##     Null deviance: 13894.3  on 82  degrees of freedom
## Residual deviance:  2387.8  on 81  degrees of freedom
##   (23 observations deleted due to missingness)
## AIC: NA
## 
## Number of Fisher Scoring iterations: 4

exp(coefficients(lag_mod))
## (Intercept)    note_lag 
##   91.816148    1.000015
exp(confint(lag_mod))
##                 2.5 %     97.5 %
## (Intercept) 80.225620 104.619006
## note_lag     1.000014   1.000017

Rsq(lag_mod)
## [1] 0.8281474

mod_fit <- as.data.frame(predict(lag_mod, type = "link", se.fit = TRUE)[1:2])

all_tot_pred <- 
  all_tot %>%
  filter(!is.na(note_lag)) %>%
  mutate(pred = mod_fit$fit,
         pred.se = mod_fit$se.fit,
         low = exp(pred - 1.96*pred.se),
         hi = exp(pred + 1.96*pred.se))


glm_fit <- all_tot_pred %>% 
    filter(!is.na(note_lag)) %>%
  ggplot(aes(x = note_lag, y = deaths)) +
  geom_point() + 
  geom_line(aes(y = exp(pred))) + 
  geom_ribbon(aes(ymin = low, ymax = hi), alpha = 0.3, col = "grey") +
  theme_bw() +
  labs(y = "Daily number of\ndeaths reported",
       x = "Daily number of NHS Pathways reports") +
  large_txt

glm_fit

4 Supplementary figures

4.1 Serial interval distribution

This is a comparison of gamma versus lognormal distribution for the serial interval used to convert r to R in our analysis. Both distributions are parameterised with mean 4.7 and standard deviation 2.9.

SI_param <- epitrix::gamma_mucv2shapescale(4.7, 2.9/4.7)
SI_distribution <- distcrete::distcrete("gamma", interval = 1,
                                        shape = SI_param$shape,
                                        scale = SI_param$scale, w = 0.5)

SI_distribution2 <- distcrete::distcrete("lnorm", interval = 1,
                                        meanlog = log(4.7),
                                        sdlog = log(2.9), w = 0.5)

SI_dist1 <- data.frame(x = SI_distribution$r(1e5)) 
SI_dist1 <- count(SI_dist1, x) %>%
    ggplot() +
    geom_col(aes(x = x, y = n)) +
    labs(x = "Serial interval (days)", y = "Frequency") +
    scale_x_continuous(breaks = seq(0, 30, 5)) +
    theme_bw()

SI_dist2 <- data.frame(x = SI_distribution2$r(1e5)) 
SI_dist2 <- count(SI_dist2, x) %>%
    ggplot() +
    geom_col(aes(x = x, y = n)) +
    labs(x = "Serial interval (days)", y = "Frequency") +
    scale_x_continuous(breaks = seq(0, 200, 20), limits = c(0, 200)) +
    theme_bw()


ggpubr::ggarrange(SI_dist1,
                  SI_dist2,
                  nrow = 1,
                  labels = "AUTO") 

4.2 Sensitivity analysis - 7 or 21 days moving window

We reproduce the window analysis with either a 7 or 21 days window for sensitivity purposes.

First with the 7 days window:

## set moving time window (1/2/3 weeks)
w <- 7

# create empty df
r_all_sliding_7days <- NULL

## make data for model
x_model_all_moving <- x %>%
  filter(!is.na(nhs_region)) %>% 
  group_by(date, nhs_region) %>%
  summarise(n = sum(count)) 

unique_dates <- unique(x_model_all_moving$date)

for (i in 1:(length(unique_dates) - w)) {
  
  date_i <- unique_dates[i]
  
  date_i_max <- date_i + w
  
  model_data <- x_model_all_moving %>%
    filter(date >= date_i & date < date_i_max) %>%
    mutate(day = as.integer(date - date_i)) %>% 
    day_of_week()
  
  
  mod <- glm(n ~ day * nhs_region + day_of_week,
             data = model_data,
             family = 'quasipoisson')
  
  # get growth rate
  r <- get_r(mod)
  r$w_min <- date_i
  r$w_max <- date_i_max
  
  # combine all estimates
  r_all_sliding_7days <- bind_rows(r_all_sliding_7days, r)
  
}

#serial interval distribution
SI_param = epitrix::gamma_mucv2shapescale(4.7, 2.9/4.7)
SI_distribution <- distcrete::distcrete("gamma", interval = 1,
                                        shape = SI_param$shape,
                                        scale = SI_param$scale,
                                        w = 0.5)

#convert growth rates r to R0
r_all_sliding_7days <- r_all_sliding_7days %>%
  mutate(R = epitrix::r2R0(r, SI_distribution),
         R_lower_95 = epitrix::r2R0(lower_95, SI_distribution),
         R_upper_95 = epitrix::r2R0(upper_95, SI_distribution))
# plot
plot_growth <-
  r_all_sliding_7days %>%
  ggplot(aes(x = w_max, y = r)) +
  geom_ribbon(aes(ymin = lower_95, ymax = upper_95, fill = nhs_region), alpha = 0.1) +
  geom_line(aes(colour = nhs_region)) +
  geom_point(aes(colour = nhs_region)) +
  geom_hline(yintercept = 0, linetype = "dashed") +
  theme_bw() +
  scale_weeks +
  theme(legend.position = "bottom",
        plot.margin = margin(0.5,1,0.5,0.5, "cm")) +
  guides(colour = guide_legend(title = "",override.aes = list(fill = NA)), fill = FALSE) +
  labs(x = "",
       y = "Estimated daily growth rate (r)") +
  scale_colour_manual(values = pal)
plot_R <- r_all_sliding_7days %>%
  ggplot(aes(x = w_max, y = R)) +
  geom_ribbon(aes(ymin = R_lower_95, ymax = R_upper_95, fill = nhs_region), alpha = 0.1) +
  geom_line(aes(colour = nhs_region)) +
  geom_point(aes(colour = nhs_region)) +
  geom_hline(yintercept = 1, linetype = "dashed") +
  theme_bw() +
  scale_weeks +
  theme(legend.position = "bottom",
        plot.margin = margin(0.5,1,0.5,0.5, "cm")) +
  guides(color = guide_legend(title = "", override.aes = list(fill = NA)), fill = FALSE) +
  labs(x = "",
       y = "Estimated effective reproduction\nnumber (Re)") +
  scale_colour_manual(values = pal)

R <- r_all_sliding_7days %>%
  mutate(lower_95 = R_lower_95, 
         upper_95 = R_upper_95,
         value = R,
         measure = "R",
         reference = 1)

r_R <- r_all_sliding_7days %>%
  mutate(measure = "r",
         value = r,
         reference = 0) %>%
  bind_rows(R)

r_R_7 <- r_R %>%
  ggplot(aes(x = w_max, y = value)) +
  geom_ribbon(aes(ymin = lower_95, ymax = upper_95, fill = nhs_region), alpha = 0.1) +
  geom_line(aes(colour = nhs_region)) +
  geom_point(aes(colour = nhs_region)) +
  geom_hline(aes(yintercept = reference), linetype = "dashed") +
  theme_bw() +
  scale_weeks +
  theme(legend.position = "bottom",
        plot.margin = margin(0.5,1,0,0, "cm"),
        strip.background = element_blank(),
        strip.placement = "outside"
  ) +
  guides(color = guide_legend(title = "", override.aes = list(fill = NA)), fill = FALSE) +
  labs(x = "", y = "") +
  scale_colour_manual(values = pal) +
  facet_grid(rows = vars(measure),
             scales = "free_y",
             switch = "y",
             labeller = as_labeller(c(r = "Daily growth rate (r)",
                                      R = "Effective reproduction\nnumber (Re)")))

Then with the 21 days window:

## set moving time window (1/2/3 weeks)
w <- 21

# create empty df
r_all_sliding_21days <- NULL

## make data for model
x_model_all_moving <- x %>%
  filter(!is.na(nhs_region)) %>% 
  group_by(date, nhs_region) %>%
  summarise(n = sum(count)) 

unique_dates <- unique(x_model_all_moving$date)

for (i in 1:(length(unique_dates) - w)) {
  
  date_i <- unique_dates[i]
  
  date_i_max <- date_i + w
  
  model_data <- x_model_all_moving %>%
    filter(date >= date_i & date < date_i_max) %>%
    mutate(day = as.integer(date - date_i)) %>% 
    day_of_week()
  
  
  mod <- glm(n ~ day * nhs_region + day_of_week,
             data = model_data,
             family = 'quasipoisson')
  
  # get growth rate
  r <- get_r(mod)
  r$w_min <- date_i
  r$w_max <- date_i_max
  
  # combine all estimates
  r_all_sliding_21days <- bind_rows(r_all_sliding_21days, r)
  
}

#serial interval distribution
SI_param = epitrix::gamma_mucv2shapescale(4.7, 2.9/4.7)
SI_distribution <- distcrete::distcrete("gamma", interval = 1,
                                        shape = SI_param$shape,
                                        scale = SI_param$scale,
                                        w = 0.5)

#convert growth rates r to R0
r_all_sliding_21days <- r_all_sliding_21days %>%
  mutate(R = epitrix::r2R0(r, SI_distribution),
         R_lower_95 = epitrix::r2R0(lower_95, SI_distribution),
         R_upper_95 = epitrix::r2R0(upper_95, SI_distribution))
# plot
plot_growth <-
  r_all_sliding_21days %>%
  ggplot(aes(x = w_max, y = r)) +
  geom_ribbon(aes(ymin = lower_95, ymax = upper_95, fill = nhs_region), alpha = 0.1) +
  geom_line(aes(colour = nhs_region)) +
  geom_point(aes(colour = nhs_region)) +
  geom_hline(yintercept = 0, linetype = "dashed") +
  theme_bw() +
  scale_weeks +
  theme(legend.position = "bottom",
        plot.margin = margin(0.5,1,0.5,0.5, "cm")) +
  guides(colour = guide_legend(title = "",override.aes = list(fill = NA)), fill = FALSE) +
  labs(x = "",
       y = "Estimated daily growth rate (r)") +
  scale_colour_manual(values = pal)
# plot
plot_R <-
  r_all_sliding_21days %>%
  ggplot(aes(x = w_max, y = R)) +
  geom_ribbon(aes(ymin = R_lower_95, ymax = R_upper_95, fill = nhs_region), alpha = 0.1) +
  geom_line(aes(colour = nhs_region)) +
  geom_point(aes(colour = nhs_region)) +
  geom_hline(yintercept = 1, linetype = "dashed") +
  theme_bw() +
  scale_weeks +
  theme(legend.position = "bottom",
        plot.margin = margin(0.5,1,0.5,0.5, "cm")) +
  guides(color = guide_legend(title = "", override.aes = list(fill = NA)), fill = FALSE) +
  labs(x = "",
       y = "Estimated effective reproduction\nnumber (Re)") +
  scale_colour_manual(values = pal)

R <- r_all_sliding_21days %>%
  mutate(lower_95 = R_lower_95, 
         upper_95 = R_upper_95,
         value = R,
         measure = "R",
         reference = 1)

r_R <- r_all_sliding_21days %>%
  mutate(measure = "r",
         value = r,
         reference = 0) %>%
  bind_rows(R)

r_R_21 <- r_R %>%
  ggplot(aes(x = w_max, y = value)) +
  geom_ribbon(aes(ymin = lower_95, ymax = upper_95, fill = nhs_region), alpha = 0.1) +
  geom_line(aes(colour = nhs_region)) +
  geom_point(aes(colour = nhs_region)) +
  geom_hline(aes(yintercept = reference), linetype = "dashed") +
  theme_bw() +
  scale_weeks +
  theme(legend.position = "bottom",
        plot.margin = margin(0.5,1,0,0, "cm"),
        strip.background = element_blank(),
        strip.placement = "outside"
  ) +
  guides(color = guide_legend(title = "", override.aes = list(fill = NA)), fill = FALSE) +
  labs(x = "", y = "") +
  scale_colour_manual(values = pal) +
  facet_grid(rows = vars(measure),
             scales = "free_y",
             switch = "y",
             labeller = as_labeller(c(r = "Daily growth rate (r)",
                                      R = "Effective reproduction\nnumber (Re)")))

And we combine both outputs into a single plot:


ggpubr::ggarrange(r_R_7,
                  r_R_21,
                  nrow = 2,
                  labels = "AUTO",
                  common.legend = TRUE,
                  legend = "bottom") 

4.3 Correlation between NHS Pathways reports and deaths by NHS region


lag_cor_reg <- data.frame()

for (i in 0:30) {

  summary <-
    all_data %>%
    group_by(nhs_region) %>%
    mutate(note_lag = lag(count, i)) %>%
    ## calculate rank correlation and bootstrap CI for each region
    group_modify(~getboot(.x)) %>%
    mutate(lag = i)
  
  lag_cor_reg <- bind_rows(lag_cor_reg, summary)
}

cor_vs_lag_reg <- 
lag_cor_reg %>%
ggplot(aes(lag, r, col = nhs_region)) +
  geom_hline(yintercept = 0, lty = "longdash") +
  geom_ribbon(aes(ymin = r_low, ymax = r_hi, col = NULL, fill = nhs_region), alpha = 0.2) +
  geom_point() +
  geom_line() +
  facet_wrap(~nhs_region) +
  scale_color_manual(values = pal) +
  scale_fill_manual(values = pal, guide = F) +  
  theme_bw() +
  labs(x = "Lag between NHS pathways and death data (days)", y = "Pearson's correlation", col = "NHS region") +
  theme(legend.position = "bottom") +
  guides(color = guide_legend(override.aes = list(fill = NA)))

cor_vs_lag_reg

5 Export data

We save the tables created during our analysis:


if (!dir.exists("excel_tables")) {
  dir.create("excel_tables")
}


## list all tables, and loop over export
tables_to_export <- c("r_all_sliding", "lag_cor")

for (e in tables_to_export) {
  rio::export(get(e),
              file.path("excel_tables",
                        paste0(e, ".xlsx")))
}

## also export result from regression on lagged data 
rio::export(lag_mod, file.path("excel_tables", "lag_mod.rds"))

6 System information

6.1 Outline

The following information documents the system on which the document was compiled.

6.2 System

This provides information on the operating system.

Sys.info()
##                                                                                            sysname 
##                                                                                           "Darwin" 
##                                                                                            release 
##                                                                                           "19.6.0" 
##                                                                                            version 
## "Darwin Kernel Version 19.6.0: Sun Jul  5 00:43:10 PDT 2020; root:xnu-6153.141.1~9/RELEASE_X86_64" 
##                                                                                           nodename 
##                                                                                "Mac-1595585185299" 
##                                                                                            machine 
##                                                                                           "x86_64" 
##                                                                                              login 
##                                                                                             "root" 
##                                                                                               user 
##                                                                                           "runner" 
##                                                                                     effective_user 
##                                                                                           "runner"

6.3 R environment

This provides information on the version of R used:

R.version
##                _                           
## platform       x86_64-apple-darwin17.0     
## arch           x86_64                      
## os             darwin17.0                  
## system         x86_64, darwin17.0          
## status                                     
## major          4                           
## minor          0.2                         
## year           2020                        
## month          06                          
## day            22                          
## svn rev        78730                       
## language       R                           
## version.string R version 4.0.2 (2020-06-22)
## nickname       Taking Off Again

6.4 R packages

This provides information on the packages used:

sessionInfo()
## R version 4.0.2 (2020-06-22)
## Platform: x86_64-apple-darwin17.0 (64-bit)
## Running under: macOS Catalina 10.15.6
## 
## Matrix products: default
## BLAS:   /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRblas.dylib
## LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib
## 
## locale:
## [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] ggnewscale_0.4.1     ggpubr_0.4.0         lubridate_1.7.9     
##  [4] chngpt_2020.5-21     cyphr_1.1.0          DT_0.14             
##  [7] kableExtra_1.1.0     janitor_2.0.1        remotes_2.2.0       
## [10] projections_0.5.1    earlyR_0.0.1         epitrix_0.2.2       
## [13] distcrete_1.0.3      incidence_1.7.1      rio_0.5.16          
## [16] reshape2_1.4.4       rvest_0.3.5          xml2_1.3.2          
## [19] linelist_0.0.40.9000 forcats_0.5.0        stringr_1.4.0       
## [22] dplyr_1.0.0          purrr_0.3.4          readr_1.3.1         
## [25] tidyr_1.1.0          tibble_3.0.3         ggplot2_3.3.2       
## [28] tidyverse_1.3.0      here_0.1             reportfactory_0.0.5 
## 
## loaded via a namespace (and not attached):
##  [1] nlme_3.1-148      fs_1.4.2          webshot_0.5.2     httr_1.4.2       
##  [5] rprojroot_1.3-2   tools_4.0.2       backports_1.1.8   utf8_1.1.4       
##  [9] R6_2.4.1          mgcv_1.8-31       DBI_1.1.0         colorspace_1.4-1 
## [13] withr_2.2.0       gridExtra_2.3     tidyselect_1.1.0  sodium_1.1       
## [17] curl_4.3          compiler_4.0.2    cli_2.0.2         labeling_0.3     
## [21] matchmaker_0.1.1  scales_1.1.1      digest_0.6.25     foreign_0.8-80   
## [25] rmarkdown_2.3     pkgconfig_2.0.3   htmltools_0.5.0   dbplyr_1.4.4     
## [29] htmlwidgets_1.5.1 rlang_0.4.7       readxl_1.3.1      rstudioapi_0.11  
## [33] farver_2.0.3      generics_0.0.2    jsonlite_1.7.0    crosstalk_1.1.0.1
## [37] car_3.0-8         zip_2.0.4         magrittr_1.5      kyotil_2019.11-22
## [41] Matrix_1.2-18     Rcpp_1.0.5        munsell_0.5.0     fansi_0.4.1      
## [45] viridis_0.5.1     abind_1.4-5       lifecycle_0.2.0   stringi_1.4.6    
## [49] yaml_2.2.1        carData_3.0-4     snakecase_0.11.0  MASS_7.3-51.6    
## [53] plyr_1.8.6        grid_4.0.2        blob_1.2.1        crayon_1.3.4     
## [57] lattice_0.20-41   cowplot_1.0.0     splines_4.0.2     haven_2.3.1      
## [61] hms_0.5.3         knitr_1.29        pillar_1.4.6      boot_1.3-25      
## [65] ggsignif_0.6.0    reprex_0.3.0      glue_1.4.1        evaluate_0.14    
## [69] data.table_1.12.8 modelr_0.1.8      vctrs_0.3.2       selectr_0.4-2    
## [73] cellranger_1.1.0  gtable_0.3.0      assertthat_0.2.1  xfun_0.15        
## [77] openxlsx_4.1.5    broom_0.7.0       rstatix_0.6.0     survival_3.1-12  
## [81] viridisLite_0.3.0 ellipsis_0.3.1